Robust self-localisation and navigation based on hippocampal place cells

A computational model of the hippocampal function in spatial learning is presented. A spatial representation is incrementally acquired during exploration. Visual and self-motion information is fed into a network of rate-coded neurons. A consistent and stable place code emerges by unsupervised Hebbian learning between place- and head direction cells. Based on this representation, goal-oriented navigation is learnt by applying a reward-based learning mechanism between the hippocampus and nucleus accumbens. The model, validated on a real and simulated robot, successfully localises itself by recalibrating its path integrator using visual input. A navigation map is learnt after about 20 trials, comparable to rats in the water maze. In contrast to previous works, this system processes realistic visual input. No compass is needed for localisation and the reward-based learning mechanism extends discrete navigation models to continuous space. The model reproduces experimental findings and suggests several neurophysiological and behavioural predictions in the rat. (c) 2005 Elsevier Ltd

Modelling Path Integrator Recalibration Using Hippocampal Place cells

The firing activities of place cells in the rat hippocampus exhibit strong correlations to the animal's location. External (e.g. visual) as well as internal (proprioceptive and vestibular) sensory information take part in controlling hippocampal place fields. Previously it has been observed that when rats shuttle between a movable origin and a fixed target the hippocampus encodes position in two different frames of reference. This paper presents a new model of hippocampal place cells that explains place coding in multiple reference frames by continuous interaction between visual and self-motion information. The model is tested using a simulated mobile robot in a real-world experimental paradigm

Minimal model of strategy switching in the plus-maze navigation task

International audiencePrefrontal cortex (PFC) has been implicated in the ability to switch behavioral strategies in response to changes in reward contingencies. A recent experimental study has shown that separate subpopulations of neurons in the prefrontal cortex were activated when rats switched between allocentric place strategies and egocentric response strategies in the plus maze. In this paper we propose a simple neural-network model of strategy switching, in which the learning of the two strategies as well as learning to select between those strategies is governed by the same temporal-difference (TD) learning algorithm. We show that the model reproduces the experimental data on both behavioral and neural levels. On the basis of our results we derive testable prediction concerning a spatial dynamics of the phasic dopamine signal in the PFC, which is thought to encode reward-prediction error in the TD-learning theory

Spontaneous Reorientation Is Guided by Perceived Surface Distance, Not by Image Matching Or Comparison

Humans and animals recover their sense of position and orientation using properties of the surface layout, but the processes underlying this ability are disputed. Although behavioral and neurophysiological experiments on animals long have suggested that reorientation depends on representations of surface distance, recent experiments on young children join experimental studies and computational models of animal navigation to suggest that reorientation depends either on processing of any continuous perceptual variables or on matching of 2D, depthless images of the landscape. We tested the surface distance hypothesis against these alternatives through studies of children, using environments whose 3D shape and 2D image properties were arranged to enhance or cancel impressions of depth. In the absence of training, children reoriented by subtle differences in perceived surface distance under conditions that challenge current models of 2D-image matching or comparison processes. We provide evidence that children’s spontaneous navigation depends on representations of 3D layout geometry.National Institutes of Health (U.S.) (Grant HD 23103

Contribution of Cerebellar Sensorimotor Adaptation to Hippocampal Spatial Memory

Complementing its primary role in motor control, cerebellar learning has also a bottom-up influence on cognitive functions, where high-level representations build up from elementary sensorimotor memories. In this paper we examine the cerebellar contribution to both procedural and declarative components of spatial cognition. To do so, we model a functional interplay between the cerebellum and the hippocampal formation during goal-oriented navigation. We reinterpret and complete existing genetic behavioural observations by means of quantitative accounts that cross-link synaptic plasticity mechanisms, single cell and population coding properties, and behavioural responses. In contrast to earlier hypotheses positing only a purely procedural impact of cerebellar adaptation deficits, our results suggest a cerebellar involvement in high-level aspects of behaviour. In particular, we propose that cerebellar learning mechanisms may influence hippocampal place fields, by contributing to the path integration process. Our simulations predict differences in place-cell discharge properties between normal mice and L7-PKCI mutant mice lacking long-term depression at cerebellar parallel fibre-Purkinje cell synapses. On the behavioural level, these results suggest that, by influencing the accuracy of hippocampal spatial codes, cerebellar deficits may impact the exploration-exploitation balance during spatial navigation

Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail

Changes of synaptic connections between neurons are thought to be the physiological basis of learning. These changes can be gated by neuromodulators that encode the presence of reward. We study a family of reward-modulated synaptic learning rules for spiking neurons on a learning task in continuous space inspired by the Morris Water maze. The synaptic update rule modifies the release probability of synaptic transmission and depends on the timing of presynaptic spike arrival, postsynaptic action potentials, as well as the membrane potential of the postsynaptic neuron. The family of learning rules includes an optimal rule derived from policy gradient methods as well as reward modulated Hebbian learning. The synaptic update rule is implemented in a population of spiking neurons using a network architecture that combines feedforward input with lateral connections. Actions are represented by a population of hypothetical action cells with strong mexican-hat connectivity and are read out at theta frequency. We show that in this architecture, a standard policy gradient rule fails to solve the Morris watermaze task, whereas a variant with a Hebbian bias can learn the task within 20 trials, consistent with experiments. This result does not depend on implementation details such as the size of the neuronal populations. Our theoretical approach shows how learning new behaviors can be linked to reward-modulated plasticity at the level of single synapses and makes predictions about the voltage and spike-timing dependence of synaptic plasticity and the influence of neuromodulators such as dopamine. It is an important step towards connecting formal theories of reinforcement learning with neuronal and synaptic properties

Spatial Learning and Action Planning in a Prefrontal Cortical Network Model

The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates

Spatial navigation deficits — overlooked cognitive marker for preclinical Alzheimer disease?

Detection of incipient Alzheimer disease (AD) pathophysiology is critical to identify preclinical individuals and target potentially disease-modifying therapies towards them. Current neuroimaging and biomarker research is strongly focused in this direction, with the aim of establishing AD fingerprints to identify individuals at high risk of developing this disease. By contrast, cognitive fingerprints for incipient AD are virtually non-existent as diagnostics and outcomes measures are still focused on episodic memory deficits as the gold standard for AD, despite their low sensitivity and specificity for identifying at-risk individuals. This Review highlights a novel feature of cognitive evaluation for incipient AD by focusing on spatial navigation and orientation deficits, which are increasingly shown to be present in at-risk individuals. Importantly, the navigation system in the brain overlaps substantially with the regions affected by AD in both animal models and humans. Notably, spatial navigation has fewer verbal, cultural and educational biases than current cognitive tests and could enable a more uniform, global approach towards cognitive fingerprints of AD and better cognitive treatment outcome measures in future multicentre trials. The current Review appraises the available evidence for spatial navigation and/or orientation deficits in preclinical, prodromal and confirmed AD and identifies research gaps and future research priorities

Fast reverse replays of recent spatiotemporal trajectories in a robotic hippocampal model

A number of computational models have recently emerged in an attempt to understand the dynamics of hippocampal replay, but there has been little progress in testing and implementing these models in real-world robotics settings. Presented here is a bioinspired hippocampal CA3 network model, that runs in real-time to produce reverse replays of recent spatiotemporal sequences in a robotic spatial navigation task. For the sake of computational efficiency, the model is composed of continuous-rate based neurons, but incorporates two biophysical properties that have recently been hypothesised to play an important role in the generation of reverse replays: intrinsic plasticity and short-term plasticity. As this model only replays recently active trajectories, it does not directly address the functional properties of reverse replay, for instance in robotic learning tasks, but it does support further investigations into how reverse replays could contribute to functional improvements